🏗 data engineering - widget101 · Scour

PySpark : The Big Brain of Data Processing 🌟spark

dev.to·2d·DEV·

What Category Theory Teaches Us About DataFrames 🧊Iceberg Tables

mchav.github.io·4d·Lobsters, Hacker News, Hacker News, r/programming·

Data Inlining in DuckLake: Unlocking Streaming for Data Lakes 🏠Data Lakehouse

ducklake.select·22h·Hacker News·

How Honeylove boosts product quality and service efficiency with BigQuery 🔍BigQuery

cloud.google.com·6h·

lynxbase/lynxdb: A lightweight schema-on-read analytics in a single binary ⚡DataFusion

github.com·2d·Hacker News·

From Pipelines to AI Platforms: How Agentic AI Is Redefining the Role of Data Engineers 🔍AI Detection

hackernoon.com·3d·

Semlib: Semantic Data Processing 🧮Apache Calcite

anishathalye.com·2d·Hacker News·

Getting Data from Multiple Sources in Power BI: A Practical Guide to Modern Data Integration for Analysts 📋CSV Processing

yourcompany.sharepoint.com·2d·DEV·

GA4 Data Quality Monitoring with BigQuery SQL ⏱️Real-time Analytics

paolobietolini.com·5d·

Show HN: Built Loony for builders who want to spin up data infrastructure fast 🧮Apache Calcite

loony.dev·6d·Hacker News·

Drizby: An Open Source BI Platform Built on a Semantic Layer (and why I built it) 🧮Apache Calcite

dev.to·17h·DEV·

Dux: Distributed DuckDB-native DataFrames for Elixir 🌟spark

dux.now·5d·Hacker News·

From SQL Analytics to Predictive Decision Systems: Operationalizing ML Models in Business Operation ⏱️Real-time Analytics

hackernoon.com·2d·

DataFuse.Net - Data Integration Framework. ⚡DataFusion

dev.to·2d·DEV·

grove/pg-trickle: A PostgreSQL extension for streaming tables with incremental view maintenance, powered by differential dataflow in Rust. 🧊Iceberg Tables

github.com·3d·Hacker News·

pg-warehouse - A local-first data warehouse at scale without over Engineering that mirrors PostgreSQL data 🏛️Lakehouse Architecture

dev.to·2d·DEV·

How to Optimize Big Data Platform Costs Across the Data Lifecycle 🗄️Storage Tiering

hackernoon.com·3d·

Show HN: Diffly – A Python package to compare polars dataframes 🐻Polars

github.com·3d·Hacker News·

How I built a data quality API that runs at the edge in milliseconds ⚡DataFusion

dev.to·3d·DEV·

Building AI Agents That Close the Loop on Pipeline Failures 🤖Copilot

hackernoon.com·4d·

Loading more...